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Abstract — Previously,  our  group  developed  autoregressive  (AR) 
models  to  predict  human  core  temperature  and  help  prevent  hyper¬ 
thermia  (temperature  >  39  °C).  However,  the  models  often  yielded 
delayed  predictions,  limiting  their  application  as  a  real-time  warn¬ 
ing  system.  To  mitigate  this  problem,  here  we  combined  AR-model 
point  estimates  with  statistically  derived  prediction  Intervals  (Pis) 
and  assessed  the  performance  of  three  new  alert  algorithms  [AR 
model  plus  PI,  median  filter  of  AR  model  plus  PI  decisions,  and 
an  adaptation  of  the  sequential  probability  ratio  test  (SPRT)].  Us¬ 
ing  field-study  data  from  22  soldiers,  including  five  subjects  who 
experienced  hyperthermia,  we  assessed  the  alert  algorithms  for 
AR-model  prediction  windows  from  15-30  min.  Cross-validation 
simulations  showed  that,  as  the  prediction  windows  increased,  im¬ 
provements  in  the  algorithms’  effective  prediction  horizons  were 
offset  by  deteriorating  accuracy,  with  a  20-min  window  providing 
a  reasonable  compromise.  Model  plus  PI  and  SPRT  yielded  the 
largest  effective  prediction  horizons  (>  18  min),  but  these  were 
offset  by  other  performance  measures.  If  high  sensitivity  and  a 
long  effective  prediction  horizon  are  desired,  model  plus  PI  pro¬ 
vides  the  best  choice,  assuming  decision  switches  can  be  tolerated. 
In  contrast,  if  a  small  number  of  decision  switches  are  desired, 
SPRT  provides  the  best  compromise  as  an  early  warning  system  of 
impending  heat  illnesses. 

Index  Terms — Autoregressive  (AR)  model,  core  temperature, 
hyperthermia,  prediction  interval  (PI),  sequential  probability  ratio 
test  (SPRT). 

I.  Introduction 

Heat  injury  is  a  problem  for  the  United  States  (U.S.) 

Armed  Forces,  especially  during  deployments  to  local¬ 
ities  with  hot  and  humid  climates,  and  trends  show  the  number 
of  heat  injury  cases  to  be  on  the  rise  each  year  [1].  From  2006 
through  2010,  there  were  2887  heat  injuries  across  the  services, 
including  311  cases  of  heat  stroke.  The  risk  of  heat  injury  is 
modulated  by  both  intrinsic  factors  (such  as  genetics,  fitness, 
acclimatization,  and  sleep  quality)  and  extrinsic  factors  (such  as 
exercise  intensity  and  duration,  clothing  and  equipment,  ambi¬ 
ent  temperature,  relative  humidity,  and  solar  radiation).  In  any 
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case,  rising  body  core  temperature  is  a  sign  of  impending  heat 
injury,  starting  with  heat  stress  (beyond  37  °C),  progressing  to 
hyperthermia  and  heat  exhaustion  (beyond  39  °C),  and  then 
to  heat  stroke  (beyond  40  °C)  [2].  The  rising  core  tempera¬ 
ture  sets  off  a  cascade  of  physiological  responses  to  preserve 
temperature  homeostasis  (~37  °C).  However,  beyond  a  critical 
temperature,  which  depends  upon  an  individual’s  intrinsic  and 
extrinsic  factors,  homeostasis  cannot  be  sustained,  leading  to 
pathophysiological  responses  that  may  culminate  in  multiorgan 
dysfunction  and  death  [2],  [3]. 

Early  recognition  of  heat  stress  combined  with  changes  of 
activity  and  cooling  strategies — such  as  cold-water  immersion, 
water  spraying,  rest  in  an  air-conditioned  area,  consumption 
of  cold  beverages,  and  ingestion  of  crushed  ice — can  reduce 
the  morbidity  and  mortality  associated  with  heat  illnesses  [4]. 
However,  certain  circumstances  make  early  recognition  and  in¬ 
tervention  challenging.  For  example,  at  the  height  of  a  military 
operation  or  during  an  athletic  competition,  soldiers  and  athletes 
may  not  perceive  the  warning  signs  of  a  rising  core  temperature 
and  impending  heat  illnesses  [5].  This  could  be  achieved  by  a 
system  capable  of  recognizing  early  trends,  reliably  predicting 
the  onset  of  core  temperature  rise,  and  generating  alerts.  Sensor 
technologies,  which  afford  the  ability  to  measure  human  core 
temperature  via  ingestible  pills  [6],  and  mathematical  predic¬ 
tive  models  [7]-[9]  could  be  coupled  to  develop  such  a  hard¬ 
ware/software  system  and  potentially  minimize  the  incidences 
of  heat  injuries. 

Data-driven  models,  such  as  artificial  neural  networks 
(ANNs)  and  autoregressive  (AR)  models,  which  use  a  time  se¬ 
ries  of  recent-past  measurements  to  predict  their  future  time 
courses,  have  been  widely  applied  for  forecasting  various  phys¬ 
ical  and  biological  signals  [  10]-[  12] .  (Note  that  AR  models  con¬ 
stitute  a  particular  case  of  ANNs  with  linear  mapping  functions.) 
In  our  earlier  work  [8],  we  used  past  core  temperature  mea¬ 
surements  to  “learn”  autocorrelations  inherently  present  in  the 
time-series  data  and  develop  AR  models  to  predict  future  core 
temperature  values  for  a  defined  prediction  window.  We  also 
computed  the  corresponding  prediction  intervals  (Pis),  which 
provide  a  measure  of  reliability  of  the  AR-model  predictions 
[13].  Importantly,  we  found  that  because  AR  models  only  de¬ 
pend  on  the  frequency  of  the  underlying  time-series  data,  and 
because  the  frequency  of  the  core-temperature  signal  is  invari¬ 
ant  from  individual  to  individual,  AR  models  could  be  consti¬ 
tuted  as  “universal”  predictors.  That  is,  once  an  AR  model  has 
learned  autocorrelations  for  one  individual,  it  can  be  directly 
used  to  predict  the  core  temperature  of  other  individuals  in  sim¬ 
ilar  populations  without  any  additional  adaption  of  the  model 
parameters. 
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However,  when  we  attempted  to  use  AR  models  for  real¬ 
time  prediction  of  core  temperature,  we  found  a  key  limitation 
[9]:  the  AR  models  introduced  significant  prediction  time  lags 
(sometimes  as  long  as  the  prediction  window  itself)  during  large 
excursions  of  the  core  temperature  signal,  which  is  exactly  the 
condition  when  we  desire  the  model  to  be  most  accurate.  Such 
prediction  time  lags  spring  from  the  observation  that  AR  mod¬ 
els  are  essentially  filters  applied  to  an  ordered  sequence  of  data 
samples  and,  hence,  yield  delayed  responses.  This  is  a  well- 
recognized  problem  also  documented  in  other  applications  of 
predictive  models  [14]-[16].  To  be  clinically  useful,  the  predic¬ 
tions  must  be  made  with  sufficient  lead-time  (effective  predic¬ 
tion  horizon:  estimated  to  be  ~20  min  in  hyperthermia  [9])  to 
allow  for  proactive  interventions  that  can  alter  the  course  of  the 
clinical  outcome. 

In  this  study,  we  aim  to  address  this  limitation,  so  as  to  al¬ 
low  for  the  practical  use  of  AR  models  as  an  effective  tool  to 
accurately  predict  core-temperature  rises  and  provide  clinically 
useful  alerts  of  impending  heat  injury.  To  this  end,  we  developed 
three  new  alert  algorithms  that  combine  AR  model  predictions 
and  associated  Pis,  and  assessed  their  ability  to  reduce  delays 
in  the  prediction  of  hyperthermia.  We  also  compared  their  per¬ 
formance  against  that  of  the  AR-model  predictions  alone.  The 
simplest  alert  algorithm  only  uses  the  current  AR-model  predic¬ 
tion  and  PI.  The  other  two  algorithms  use  a  series  of  recent-past 
predictions  and  Pis  to  make  the  decision.  Of  these,  one  alert  al¬ 
gorithm  provides  explicit,  adjustable  parameters,  separate  from 
the  AR-model  parameters,  to  tradeoff  conflicting  measures  of 
performance. 

To  evaluate  the  performance  of  these  algorithms,  we  used 
field-study  data  from  22  subjects  involved  in  military  activities, 
including  five  subjects  whose  core  temperature  rose  beyond 
39  °C,  the  onset  of  hyperthermia.  We  assessed  the  algorithms 
against  four  measures  of  performance,  namely,  effective  predic¬ 
tion  horizon,  sensitivity,  specificity,  and  decision  switches.  Two 
algorithms  reliably  predicted  the  onset  of  hyperthermia  ~20  min 
in  advance  with  reasonably  high  sensitivity  and  specificity,  with 
one  of  them  yielding  fewer  decision  switches.  These  promising 
results  form  the  basis  for  real-time  core  temperature  predictions, 
which,  when  coupled  with  temperature  sensors,  could  provide 
reliable  early  warning  of  impending  heat  illnesses. 

II.  Methods 

A.  AR  Model 

Given  temperature  measurements  yn-i  sampled  every  S  min, 
where  n  is  the  current  discrete  time  index  and  z  =  0,  1,  ...  , 
m  -  I,  the  AR  model  of  order  m  predicts  signal  at  time 

point  n  +  1,  through  a  linear  combination  of  the  antecedent 
core-temperature  samples  as  follows: 

m  —  1 

Vn  +  l  ^  ^  hiyn-^  (1) 

where  b  denotes  the  vector  of  m  unknown  AR  coefficients. 
To  make  predictions  M  time  steps  ahead  (prediction  window 
P  =  M  X  S  min  ahead),  we  iteratively  used  (1)  M  times,  sub¬ 


stituting  the  unobserved  signals  at  n  >  n+ 1  in  the  summation  by 
their  corresponding  predicted  values.  The  order  m  of  the  model 
specifies  the  required  initial  waiting  period  for  which  data  sam¬ 
ples  need  to  be  collected  before  real-time  predictions  can  be 
made.  We  chose  the  sampling  period  S  that  preserved  the  im¬ 
portant  frequencies  and  rejected  the  high-frequency  noise  in  the 
magnitude  spectrum  of  the  core-temperature  data  [17].  Subse¬ 
quently,  we  chose  m  to  be  the  number  of  lags  in  the  data  beyond 
which  the  partial  autocorrelation  function  was  essentially  zero 
[10]. 

Before  applying  the  AR  model,  we  must  first  estimate  the 
AR  coefficients  b  using  some  “training”  data.  To  estimate  b,  we 
used  the  standard  forward-backward  least  squares  procedure 
(see  [17,  Ch.  8])  implemented  in  MATLAB  version  7.14  (func¬ 
tion  ar).  In  this  procedure,  we  first  used  the  entire  time  series  of 
temperature  measurements  to  form  a  time-forward  convolution 
matrix,  and  then  a  time-backward  convolution  matrix.  Next, 
we  combined  the  two  matrices  into  one,  estimated  the  corre¬ 
sponding  autocorrelation  matrix,  and  inverted  it  to  estimate  the 
coefficients  b.  An  advantage  of  the  forward-backward  proce¬ 
dure  is  that  it  implicitly  regularizes  the  autocorrelation  matrix 
(which  can  be  ill  conditioned  due  to  noise  in  the  data)  to  ensure 
robust  estimates  of  b. 

In  many  safety-critical  applications,  providing  single-point 
temperature  predictions  may  not  be  sufficient  for  decision  mak¬ 
ing  and  may  require  information  about  the  uncertainty  of  the 
predicted  values.  In  earlier  work  [18],  we  developed  a  tech¬ 
nique  based  on  the  statistical  bootstrap  method  [19]  to  estimate 
prediction  uncertainty  in  the  form  of  Pis.  The  technique  relies  on 
the  idea  of  model  resampling  [13]  rather  than  data  resampling, 
where  a  population  of  models  is  built  based  on  blocks  of  data 
that  are  randomly  drawn  from  the  original  time  series  to  form  an 
empirical  distribution  of  models  (i.e.,  a  distribution  of  the  model 
coefficients).  Following  this  procedure  [18]  to  compute  the  Pis, 
we  estimated  the  covariance  matrix  S  of  the  AR  model  from  a 
distribution  of  models  for  an  M  time-step-ahead  predictor  and 
used  the  following  equation  [18]: 

PI  =  Z„/2Vy^S^+^  (2) 

where  Z^i2  denotes  the  prediction  factor  associated  with  an 
a%  type  I  error,  y  represents  a  vector  of  data  samples  y  = 
[t/n  2/n-i  •  •  ■  yn-m+i  and  (T^  denotes  the  variance  of  the 
measurement  noise.  We  used  the  AR  model  in  (1)  to  make  M 
step-ahead  predictions  (P-min-ahead)  of  core  temperature  and 
(2)  to  estimate  the  Pis. 

As  mentioned  earlier,  we  found  that  AR  models  introduce 
significant  prediction  time  lags  during  large  excursions  of  the 
core  temperature  signal  [9].  This  significantly  reduces  the  ef¬ 
fective  prediction  horizon  whenever  there  is  a  steep  rise  in  core 
temperature,  which  is  the  case  when  the  temperature  nears  the 
hyperthermic  threshold  of  39  °C.  To  address  this  limitation,  we 
investigated  three  alert  algorithms  that  combine  AR  model  pre¬ 
dictions  and  associated  Pis  to  determine  whether  the  predicted 
core  temperature  exceeded  a  predefined  threshold  and  compared 
their  performance  to  the  AR-model  predictions  alone. 
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B.  Sequential  Probability  Ratio  Test  (SPRT) 

Bayesian  approaches,  which  use  information  from  previous 
data  (previous  predictions  in  this  case)  and  current  observations 
(current  predictions),  offer  a  natural  choice  for  this  decision¬ 
making  problem.  The  SPRT  [20]  is  a  Bayesian  approach  that 
considers  increasing  evidence  from  a  sequence  of  observations 
to  make  a  decision  [21],  [22].  Briefly,  given  a  sequence  of  core¬ 
temperature  samples  Xi,  X2,  . . . ,  not  necessarily  independent, 
so  that  X  ^  JV{^x  ,o-x)  ^  normal  Gaussian  process  with 

unknown  mean  fix  and  a  given  variance  cr^,  the  SPRT  tests 
the  null  hypothesis  (Hg)  that  fix  =  Mo  against  an  alternative 
hypothesis  (Hi )  that  fix  =  fii,  where  fiQ  and  fii  denote  the  mean 
temperature  values  below  and  above  the  temperature  threshold, 
respectively,  with  fiQ  <  /ii-  If  Po  and  pi  are  the  probability 
density  functions  governing  Hg  and  Hi,  respectively,  then  the 
observed  likelihood  ratio  at  decision  time  t  (corresponding  to  the 

K-l  , 

time  point  n)  can  be  represented  as  In  =  l  )  ’  where 

A:=0 

K  is  the  length  of  the  sequence  of  samples  being  considered. 

In  order  to  apply  the  SPRT  algorithm  to  our  problem,  we 
combined  three  predicted  values,  namely,  the  AR-model  pre¬ 
diction  (y„_fc),  the  upper  PI  {yn~k  +  PIn-fc),  and  the  lower  PI 
iVn-k  —  Plyi-fc),  using  Weights  9  and  </)  to  form: 

Xn^k  =  Vn-k  -  PIn-fc  {I  -  9(j)  -  (j))  (3) 

where  fc  =  (0,  1,  . . . ,  K  —  1)  denotes  a  time  index  and  the 
weights  6  and  ip  are  constrained  to  be  between  0  and  1 .  In  (3), 
when  (p  =  0,  Xn^k  equals  the  lower  PI;  when  </>  =  1  and  9  = 
0,  Xn^k  equals  the  AR-model  prediction;  and  when  9  =  <j)=  1, 
Xn-k  equals  the  upper  PI.  Thus,  lies  between  the  lower  and 
upper  Pis  for  all  values  of  9  and  p.  Note  that  a  decision  at  time  t, 
for  a  prediction  window  of  P  min,  is  actually  made  at  time  t-P, 
which  corresponds  to  time  point  n-M  for  M  steps-ahead  AR- 
model  predictions.  Then,  following  Wald’s  SPRT  methodology 
[20],  we 

accepted  Hg  (below  temperature  threshold) 
iflog((„)  <  log(H);or 
accepted  Hi  (above  temperature  threshold) 

iflog((„)  <  log(A);  or  ^ 

made  no  decision  and  proceeded  to  time  n+1 
if  log(H)  <  log(/„)  <  log(A) 

where  A  and  B  are  constants  that  control  the  false-positive  rate 
and  false-negative  rate,  respectively,  with  0  <  H  <  A  <  oo. 
The  SPRT  algorithm  required  the  estimation  of  seven  param¬ 
eters,  the  two  weights  9  and  p  in  (3)  and  the  five  parameters 
fiQ,  Ml ,  cTx,  A,  and  B,  from  a  subject’s  core-temperature  data. 
A  large  difference  between  fiQ  and  fii ,  with  a  small  ax  (which 
depends  on  the  measurement  noise),  leads  to  good  separabil¬ 
ity  between  temperature  values  below  and  above  the  threshold, 
respectively.  This  leads  to  a  significant  increase  or  decrease  in 
log((„),  depending  on  whether  the  AR  model  tracks  a  rise  or  fall 
in  the  core  temperature  data,  respectively.  A  large  difference 
between  log(A)  and  log(B)  ensures  the  reduction  of  false  alerts 
while  maintaining  the  sensitivity  of  the  algorithm. 


C.  Proposed  Alert  Algorithms 

As  mentioned  earlier,  AR  model  predictions  are  significantly 
delayed  during  steep  rises  in  core  temperature.  To  address  this 
problem  and  provide  an  alert  before  the  core  temperature  reaches 
a  specified  threshold,  we  compared  the  performance  of  three 
algorithms,  which  use  AR-model  predictions  and  Pis,  against 
the  AR  model.  Of  these  algorithms,  the  SPRT  is  the  only  one 
that  required  estimation  of  additional  parameters.  All  algorithms 
output  either  0  or  1  (no  alert  or  alert,  respectively).  The  four 
algorithms  are  as  follows: 

1)  Model:  Uses  the  AR-model  prediction  at  the  current  time 
instant  to  make  a  decision. 

2)  Model+PI:  Uses  the  upper  PI  (AR-model  predictions  plus 
PI)  at  the  current  time  instant  to  make  a  decision. 

3)  Median  Filter:  Outputs  the  median  of  a  finite  sequence  of 
decisions  made  by  Model+PI.  The  length  of  the  sequence 
of  decisions  was  fixed  to  an  odd  number  (five)  to  ensure 
that  the  output  was  either  0  or  1 . 

4)  SPRT:  Outputs  a  decision  based  on  (4)  after  collecting 
evidence  from  a  series  of  AR-model  predictions  and  Pis 
[see  (3)].  For  a  given  temperature  threshold,  we  obtained 
the  SPRT  parameters  by  minimizing  a  cost  function  [see 
Appendix  A,  (Al)]  formed  as  a  composite  of  the  four 
measures  of  performance  described  in  Section  II-D. 

D.  Measures  for  Evaluating  the  Performance 
of  the  Proposed  Algorithms 

To  evaluate  the  proposed  alert  algorithms,  we  defined  an 
“event”  as  an  episode  where  the  core  temperature  measurement 
rises  and  remains  above  a  specified  temperature  threshold  for 
>15  min  [see  Fig.  1(a)].  The  event  ends  when  the  measured 
temperature  decreases  below  the  threshold  and  remains  below 
the  threshold  for  >15  min.  Thus,  an  event  was  mapped  into  a  1 
(true  response)  when  the  measured  temperature  was  above  the 
threshold  and  0  otherwise.  Similarly,  a  model-predicted  event 
was  defined  as  an  episode  where  the  algorithm’s  output  (the 
model-predicted  response)  was  1  [see  Fig.  1(a)]. 

We  evaluated  the  algorithms’  performance  using  four  mea¬ 
sures: 

1)  Sensitivity,  the  fraction  of  time  points  during  which 
both  the  true  and  the  model-predicted  responses  were  1 
[Fig.  1(c);  involving  time  spans  A  and  B],  including  the 
time  points  up  to  at  most  30  min  prior  to  the  true  event 
onset  when  the  model-predicted  responses  were  1  and  the 
true  responses  were  0  [Fig.  1(c);  scenario  ii].  This  al¬ 
lowed  us  to  reward  model-predicted  events  that  arrived  up 
to  30  min  before  the  true  event  onset,  as  it  takes  >30  min 
for  the  core  temperature  to  rise  by  1  °C  during  exercise 
[23].  Sensitivity  was  expressed  as  a  percentage  of  time 
points  of  the  entire  time  series. 

2)  Specificity,  the  fraction  of  time  points  during  which 
both  the  true  and  the  model-predicted  responses  were  0 
[Fig.  1(c)  and  (d);  involving  time  spans  C  and  D].  For 
computing  specificity,  we  did  not  consider  time  points  up 
to  at  most  30  min  prior  to  the  true  event  onset  and  time 
points  immediately  following  the  true  event,  when  the 
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Fig.  1.  Definition  of  a  true  event  and  the  four  measures  of  algorithm  performance.  These  measures  compare  a  true  event,  i.e.,  the  episode  when  the  core 
temperature  rose  above  a  certain  temperature  threshold,  with  the  model-predicted  event,  (a)  Measurement  data  (dotted  line)  and  P-min-ahead  AR-model 
predictions  (dashed  line).  In  this  example,  the  AR-model  used  data  until  time  point  t  to  make  a  prediction  at  time  point  t  +  P.  (b)  Binary  responses  of 
the  true  event  (solid  line)  and  the  model-predicted  event  (dashed  line).  The  effective  prediction  horizon  was  defined  as  the  prediction  window  P  plus  the 
time  difference  At  between  the  onsets  of  the  true  and  model-predicted  events  (see  text  for  complete  definition),  (c)  In  scenario  i,  the  prediction  was  de¬ 
layed  (At  <  0)  and  in  scenario  ii,  it  anticipated  the  true  event  (At  >  0).  We  computed  sensitivity  and  specificity  using  time  markers  A,  B,  C,  and  D. 
Sensitivity  was  defined  as  the  fraction  of  B  included  in  max(A,  B)  (<100%  in  scenario  i  and  100%  in  scenario  ii).  Specificity  was  defined  as  the  frac¬ 
tion  of  C  included  in  max(C,  D-At)  if  At  >  0  and  the  fraction  of  C  included  in  max(C,  D)  otherwise,  (d)  Scenario  where  there  was  no  true  event 
but  the  model  predicted  an  event.  In  this  case,  we  could  only  compute  specificity  and  the  number  of  decision  switches;  the  other  two  measures  were 
undefined. 


model-predicted  responses  were  1  and  the  true  responses 
were  0  [Fig.  1(c);  scenario  ii].  Specificity  was  expressed 
as  a  percentage  of  time  points  of  the  entire  time  series. 

3)  Effective  prediction  horizon:  For  each  true  event,  we  com¬ 
puted  the  effective  prediction  horizon  hy  adding  the  pre¬ 
diction  window  P  to  the  time  difference  At  between  the 
onset  of  the  true  event  and  the  onset  of  the  model-predicted 
event.  When  At  <  0  [Fig.  1(b)  and  (c);  scenario  i],  the  on¬ 
set  of  the  true  event  preceded  the  prediction  and  the  lower 
limit  of  At  was  set  to -F  min.  When  At  >  0  [Fig.  1(c);  sce¬ 
nario  ii],  the  onset  of  the  model-predicted  event  preceded 
the  true  event  and  the  upper  limit  of  At  was  set  to  30  min 
(as  explained  earlier).  The  reported  effective  prediction 
horizon  was  averaged  over  the  number  of  events. 

4)  Number  of  decision  switches:  the  cumulative  number  of 
times  the  model-predicted  output  transitioned  from  one 
state  to  another  (0  to  1  or  1  to  0)  that  was  incongruent 
with  the  true  state  (0  or  1)  of  the  measured  temperature 
[Fig.  1(d)  shows  two  decision  switches:  the  true  state  was 
0  and  the  model-predicted  output  changed  first  from  0  to 
1  and  then  from  1  to  0]. 

We  computed  specificity  and  the  number  of  decision  switches 
regardless  of  whether  or  not  an  event  had  occurred;  however,  we 
only  computed  sensitivity  and  effective  prediction  horizon  when 
a  true  event  occurred.  Note  that  while  specificity  measured  the 
time  period  for  which  the  algorithm  incorrectly  predicted  the 
occurrence  of  an  event,  the  number  of  decision  switches  pro¬ 
vided  the  number  of  times  the  algorithm  predictions  incorrectly 
switched  from  one  state  to  another. 


E.  Study  Data 

To  demonstrate  the  performance  of  the  proposed  algorithms, 
we  used  data  from  a  field  study  involving  22  U.S.  Army  soldiers 
[age:  23.1  year  (SD  4.1);  height:  178  cm  (SD  7);  weight  81.3  kg 
(SD  11.1),  mean  and  standard  deviation  (SD)]  who  performed 
regularly  scheduled  infantry  training.  The  training  included  a  6- 
mile  foot  march  while  wearing  a  backpack  and  carrying  equip¬ 
ment  weighing  on  average  14.0  kg  (SD  1.4)  and  exercises,  such 
as  digging  of  ditches,  setting  up  concertina  wire,  marksmanship 
drills,  running,  rolling,  and  jumping  as  part  of  approach  to  a 
target.  The  training  duration  was  8-14  h  in  one  day.  During 
the  training,  soldiers  wore  the  advanced  combat  uniform  with 
a  thermal  insulation  of  1.08  do  and  an  evaporative  potential 
of  0.41  im/clo.  The  ambient  temperature  was  31.5  °C  (SD  2.9) 
with  a  relative  humidity  of  66%  (SD  16)  and  a  wind  speed  of 
9.9  km/h  (SD  3.7).  The  Institutional  Review  Board  of  the  U.S. 
Army  Research  Institute  of  Environmental  Medicine  (Natick, 
MA)  approved  the  study.  Subjects  were  briefed  on  the  purpose, 
risks,  and  benefits  of  the  study  and  each  gave  their  written  in¬ 
formed  consent  prior  to  study  participation.  All  training  was  at 
the  direction  of  the  military  unit,  i.e.,  the  research  team  did  not 
interfere  with  or  ask  for  any  alteration  to  training  events;  they 
only  monitored  physiological  parameters  of  the  subjects.  The 
core  temperature  data  were  measured  using  radio-thermometer 
pills  (MiniMitter,  Inc.,  Bend,  OR)  that  transmitted  the  data  to 
the  Hidalgo  Equivital  EQ-02  (Hidalgo,  Ltd.,  Cambridge,  UK) 
physiological  status  monitoring  (PSM)  system  [24].  Data  were 
retrieved  from  the  PSM  system  at  the  end  of  the  exercise  and 
subsequently  analyzed.  Pills  were  ingested  at  least  12  h  before 
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data  collection  and  had  the  following  technical  characteristics: 
size:  21.9  mm  length  and  8.5  mm  diameter;  weight:  1.75  g;  sam¬ 
pling  period:  15  s;  temperature  range:  25-50  °C,  with  accuracy 
of  ±0.25  °C;  transmission  method:  near-field  magnetic  link. 

F.  Cross  Validation  of  the  Algorithms 

We  performed  a  cross  validation  of  the  four  alert  algorithms 
using  the  study  data  as  follows: 

1)  Step  1:  Identified  subjects  that  had  core  temperature  val¬ 
ues  above  a  38  °C  temperature  threshold  for  >  15  min. 
Eighteen  subjects  met  this  criterion. 

2)  Step  2:  Trained  an  AR  model  and  estimated  the  SPRT 
algorithm  parameters  using  data  from  one  of  the  subjects 
who  met  the  criterion  in  Step  1 .  Tested  the  algorithms  on 
the  remaining  21  subjects  by  computing  the  four  measures 
of  performance  (sensitivity,  specificity,  effective  predic¬ 
tion  horizon,  and  the  number  of  decision  switches)  for 
each  algorithm.  Repeated  this  step  for  each  of  the  sub¬ 
jects  who  met  the  criterion  in  Step  1. 

3)  Step  3:  Increased  the  temperature  threshold  by  0. 1  °C  and 
repeated  Steps  1  and  2. 

This  procedure  was  repeated  for  temperature  thresholds  from 
38.0  to  39.5  °C.  There  were  five  subjects  with  core  temperature 
values  beyond  39  °C  (hyperthermic  threshold)  and  three  subjects 
with  core  temperature  values  beyond  39.5  °C  (see  Table  III  in 
Appendix  B). 


III.  Results  and  Discussion 

To  apply  the  AR  model  to  the  core  temperature  data,  we  esti¬ 
mated  the  sampling  period  S  and  the  model  order  m  as  follows. 
Analysis  of  the  magnitude  spectrum  of  the  core  temperature  data 
led  us  to  fix  5  to  5  min  to  preserve  the  important  frequencies  of 
the  signal  while  rejecting  noise.  Accordingly,  the  temperature 
measurements  were  downsampled  and  reported  every  5  min. 
Subsequently,  we  fixed  m  to  5  based  on  analysis  of  the  partial 
autocorrelation  function.  For  computing  the  PI  in  (2),  we  set 
Z^I2  to  2.78  based  on  the  t-distribution  for  a  =  5%  and  m  = 
5  [18].  To  further  test  the  universal  nature  of  our  AR  models 
[8],  for  each  subject,  we  compared  the  root  mean  squared  error 
(RMSE)  between  the  AR-model  fit  and  the  measured  data  with 
the  average  RMSE  between  each  of  the  other  21  AR-model 
predictions  and  the  measured  data  for  that  subject.  We  found 
that  the  differences  between  the  22  pairs  of  RMSEs  were  not 
significantly  different  (p  =  0.55,  paired  Wilcoxon  signed-rank 
test)  and  the  mean  RMSE  difference  was  0.05  °C  (SD  0.05  °C). 
For  the  SPRT  algorithm,  the  memory  length  K  was  fixed  to  3 
(corresponding  to  a  memory  of  15  min  for  S  =  5  min). 

A.  Performance  of  the  Algorithms  for  a  Single 
Temperature  Threshold 

Fig.  2  shows  the  performance  of  the  four  alert  algorithms  on 
a  single  subject  based  on  a  temperature  threshold  of  39  °C 
using  a  model  trained  on  data  from  another  subject  (see 
Section  II-F).  The  top  panel  shows  the  measured  core  temper¬ 
ature  data  (dotted  line),  20-min-ahead  AR-model  predictions 
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Fig.  2.  Top  panel  shows  the  measured  core  temperature  data  from  one  subject, 
20-min-ahead  AR  model  predictions,  and  the  corresponding  Pis.  The  next  four 
panels  show  the  depiction  of  a  true  event  (solid  line)  and  algorithm  decisions 
(dashed  lines),  for  Model,  Model+PI,  Median  Filter,  and  SPRT  algorithms, 
respectively.  The  vertical  thin  dotted  line  in  these  four  panels  marks  the  30-min 
point  that  precedes  the  true  event  onset. 


(dashed  line),  and  95%  Pis  (shaded  area).  The  next  four  panels 
show  the  depiction  of  a  true  event  (solid  line)  and  algorithm 
decisions  (dashed  lines),  for  Model,  Model±PI,  Median  Filter, 
and  SPRT  algorithms,  respectively.  The  vertical  thin  dotted  line 
in  these  four  panels  marks  the  30-min  point  that  precedes  the 
true  event  onset. 

As  shown  in  Fig.  2  (top  panel),  we  observed  that  the  up¬ 
per  PI  leads  the  measured  data,  whereas  the  model  predictions 
lag  the  data.  Hence,  by  construction.  Model  yielded  the  largest 
specificity,  the  smallest  sensitivity,  and  the  shortest  effective 
prediction  horizon  compared  with  any  of  the  other  three  algo¬ 
rithms.  Model±PI  yielded  the  largest  sensitivity  (100%)  and 
the  longest  effective  prediction  horizon  (30  min;  the  predicted 
event  anticipated  the  actual  event  by  10  min  beyond  the  20-min 
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Fig.  3.  Cross-validation  study  of  the  four  algorithms  across  temperature  thresholds  from  38.0  to  39.5  “C.  Plots  show  the  mean  (standard  error)  across  the  subjects 
for  each  of  the  four  performance  measures  for  20-min-ahead  AR-model  predictions.  Sensitivity  and  effective  prediction  horizon  were  defined  only  for  subjects 
with  events,  i.e.,  those  whose  core  temperature  exceeded  the  corresponding  temperature  threshold. 


prediction  window).  However,  due  to  inherent  noise  in  the  data, 
Model+PI  also  yielded  a  large  number  of  false  alerts,  leading 
to  reduced  specificity  and  a  large  number  of  decision  switches 
compared  to  Model  (specificity:  97%  versus  100%;  number  of 
decision  switches:  6  versus  0). 

When  compared  to  the  decisions  of  Model+PI,  Median  Filter 
yielded  fewer  decision  switches  at  the  cost  of  a  reduced  effective 
prediction  horizon  (number  of  decision  switches:  2  versus  6;  ef¬ 
fective  prediction  horizon:  20  min  versus  30  min).  SPRT  had 
the  same  effective  prediction  horizon  as  Median  Filter  (20  min) 
but,  importantly,  had  no  decision  switches  (0  versus  2).  A  four¬ 
fold  cross-validation  study  of  five  subjects  whose  temperature 
exceeded  39  °C  in  eight  events  (Table  III)  confirmed  the  afore¬ 
mentioned  findings  to  hold  true  across  our  dataset  (see  Table  I). 
In  these  five  sets  of  computations,  the  range  of  the  SPRT  param¬ 
eters  obtained  by  optimizing  (Al)  was  as  follows:  9  (0.80-0.8 1), 
(j)  (0.71-0.80),  no  (38.0-38.4),  ni  (40.0-40.2),  cr^  (0.20-0.25), 
log(A)  (—89  to  76),  and  log(B)  (—95  to  19).  The  large  range  of 
values  for  log(A)  and  log(B)  was  driven  by  one  subject  [#11; 
log(A)  =  76  and  log(Z?)  =  19],  who  experienced  temperature 
excursions  just  below  the  39  °C  threshold,  and  by  the  goal  of 
the  parameter  optimization  routine  to  reduce  the  number  of  false 


TABLE  I 

Cross-Validation  Analysis  for  20-min-ahead  AR-Model  Predictions 
AT  A  Temperature  Threshold  of  39  “C 


Algorithm 

Sensitivity 

(%) 

Specificity 

(%) 

Effective 
prediction 
horizon  (min) 

Number  of 
decision 
switches 

Model 

74.0(1.7) 

99.3  (0.3) 

8(1) 

2 

Model+PI 

97.6  (0.7) 

94.1  (1.3) 

29(1) 

8 

Median  Filter 

93.6(1.2) 

95.1  (1.3) 

19(1) 

2 

SPRT 

88.3  (3.6) 

96.3  (1.4) 

18(3) 

2 

Values  are  mean  (standard  error)  from  a  fourfold  cross-validation  study  of  five  subjects 
whose  core  temperatures  exceeded  39  °C  in  eight  distinct  events  (see  Table  III  in 
Appendix  B). 


alerts,  which  tended  to  maximize  the  differences  between  these 
two  parameters. 

B.  Performance  of  the  Algorithms  for  Temperature  Thresholds 
From  38.0  to  39.5  °C 

Fig.  3  shows  the  overall  cross-validation  results  of  each  of 
the  four  algorithms  for  20-min-ahead  AR-model  predictions  for 
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temperature  thresholds  between  38.0  and  39.5  °C,  for  every 

O.  1  °C  increment.  Due  to  varying  numbers  of  subjects  with  core 
temperature  values  beyond  a  specified  temperature  threshold 
(see  Appendix  B,  Table  111),  the  number  of  subjects  from  which 
we  computed  sensitivity  and  effective  prediction  horizon  varied. 
For  all  temperature  thresholds,  we  computed  sensitivity  and 
effective  prediction  horizon  using  fewer  subjects  than  we  used 
to  compute  specificity  and  the  number  of  decision  switches, 
which  was  calculated  for  each  of  the  22  subjects. 

Similar  to  the  example  in  Fig.  2,  Model  (see  Fig.  3,  squares) 
yielded  the  smallest  sensitivity,  the  shortest  effective  predic¬ 
tion  horizon,  and  the  largest  specificity  across  all  temperature 
thresholds.  Model+Pl  (see  Fig.  3,  circles)  yielded  high  sensi¬ 
tivity  and  a  long  effective  prediction  horizon,  while  producing 
a  very  large  number  of  decision  switches  across  all  temperature 
thresholds.  Median  Filter’s  performance  (see  Fig.  3,  triangles) 
was  generally  between  that  of  Model-bPl  and  Model  across  all 
measures.  In  contrast,  the  SPRT  algorithm  (see  Fig.  3,  diamonds) 
yielded  varying  performance  across  the  temperature  thresholds, 
partially  because  we  estimated  different  SPRT  parameters  for 
each  temperature  threshold  (see  Section  11-C). 

In  Fig.  3  (right  panels),  the  specificity  of  Model+PI  increased 
and  the  number  of  decision  switches  decreased  with  increasing 
temperature  thresholds.  This  was  because  the  sustained  steep 
rise  in  the  temperature  data  reduced  the  effects  of  measurement 
noise.  This  allowed  the  AR  model  to  better  track  the  data,  leading 
to  fewer  prediction  errors  for  Model+PI  at  higher  temperature 
thresholds. 

The  SPRT  algorithm  yielded  sensitivities  around  90%  for 
most  temperature  thresholds  and  yielded  an  overall  increas¬ 
ing  trend  in  the  effective  prediction  horizon  with  increasing 
thresholds,  especially  beyond  39  °C.  This  is  because  at  higher 
thresholds  pi  (the  mean  temperature  above  the  threshold)  was 
significantly  higher  than  p,o  (the  mean  temperature  below  the 
threshold),  leading  to  a  wider  separation  between  core  temper¬ 
ature  values  above  and  below  the  temperature  threshold  and 
improved  estimates  of  A  and  B  [see  (4)],  which  reduced  the 
false-positive  rate  and  the  false-negative  rate. 

C.  Performance  of  the  Algorithms  as  a  Function 
of  the  AR  Model  Prediction  Window 

To  evaluate  the  performance  of  the  algorithms  as  a  function 
of  prediction  window  of  the  AR  model,  we  set  P  to  15,  20,  and 
30  min  and  repeated  the  cross-validation  study  of  the  four  algo¬ 
rithms  for  hyperthermic  temperature  thresholds  (39.0-39.5  °C). 
For  all  algorithms,  sensitivity  and  specificity  decreased,  while 
the  effective  prediction  horizon  increased  with  increasing  P  (see 
Table  II).  For  all  algorithms,  and  in  particular  Model+PI,  the 
number  of  decision  switches  tended  to  increase  with  increasing 

P.  SPRT  yielded  effective  prediction  horizons  comparable  to 
Model+PI  for  P  >  20  min  (see  Table  II,  P  =  20:  23  versus 
25',  P  =  30:  32  versus  30,  all  units  in  min)  with  a  significantly 
lower  number  of  decision  switches. 

The  cross-validation  studies  also  suggested  that,  in  general, 
the  SPRT  parameters  estimated  from  one  subject’s  data  could 
be  directly  applied  to  other  subjects  for  the  same  temperature 


TABLE  II 

Cross-Validation  Study  of  the  Four  Algorithms  as  a  Function 
OF  AR-Model  Prediction  Window  for  Hyperthermic 
Temperature  Thresholds  (39.0-39.5  °C) 


Algorithm 

Sensitivity^ 

(%) 

Specificity’^ 

(%) 

Effective 
prediction 
horizon^  (min) 

Number  of 
decision 
switches’" 

15  min-ahead  AR-model  predictions 

Model 

80.7  (2.6) 

99.6  (0.0) 

9(1) 

1 

Model+PI 

98.7  (0.7) 

95.5  (0.0) 

23(1) 

4 

Median  Filter 

93.1  (1.9) 

96.1  (0.0) 

13(1) 

2 

SPRT 

93.8  (2.3) 

95.6  (O.I) 

18(3) 

2 

20  min- 

•ahead  AR-model  predictions 

Model 

70.3  (2.5) 

98.9  (0.0) 

9(1) 

1 

Model+PI 

97.4(1.2) 

94.9  (0.0) 

25(1) 

5 

Median  Filter 

87.0  (2.2) 

95.7  (O.I) 

15(1) 

2 

SPRT 

92.7  (2.7) 

94.3(0.1) 

23  (3) 

2 

30  min-ahead  AR-model  predictions 

Model 

51.5  (2.6) 

99.2  (0.0) 

11(1) 

2 

Model+PI 

88.1  (1.8) 

93.4(0.1) 

30  (2) 

7 

Median  Filter 

69.7  (3.3) 

94.4  (0.1) 

19(2) 

2 

SPRT 

87.8  (4.2) 

91.6  (0.2) 

32(4) 

3 

Table  shows  values  averaged  over  temperature  thresholds  from  39.0  to  39.5  °C. 

*Values  are  mean  (standard  error)  across  all  22  subjects  in  the  cross-validation  study 
(see  Table  III  in  Appendix  B). 

^Values  are  mean  (standard  error)  across  subjects  with  events  in  the  cross-validation 
study  (see  Table  III  in  Appendix  B). 

threshold.  However,  we  obtained  more  robust  parameter  esti¬ 
mates,  in  particular  for  parameters  A  and  B,  when  the  training 
subject’s  data  both  exceeded  the  temperature  threshold  and  hov¬ 
ered  just  below  it.  Using  such  a  procedure  to  select  subjects  from 
which  to  estimate  the  SPRT  parameters,  as  opposed  to  using 
all  subjects  whose  temperature  exceeded  the  threshold  as  per¬ 
formed  here,  would  have  improved  the  specificity  of  the  SPRT 
algorithm  in  Tables  I  and  II. 

IV.  Conclusion 

We  found  that,  as  expected,  using  Pis  along  with  AR-model 
predictions  increased  the  effective  prediction  horizon,  enabling 
earlier  detection  of  the  onset  of  core  temperature  rise  than  other¬ 
wise  possible  using  AR-model  predictions  alone.  We  also  found 
that  none  of  the  three  proposed  alert  algorithms  was  consistently 
superior  in  each  of  the  four  assessed  measures  of  performance. 
While  Model+PI  yielded  the  largest  sensitivity  and  the  longest 
effective  prediction  horizon,  it  also  yielded  the  largest  number 
of  decision  switches.  In  contrast,  delaying  alert  decisions — 
through  filtering  (Median  Filter)  or  through  accumulation  of 
evidence  (SPRT) — yielded  a  small  number  of  decision  switches 
at  the  cost  of  reduced  sensitivity  and  effective  prediction 
horizon. 

Increasing  the  prediction  window  of  the  AR  model  increased 
the  effective  prediction  horizon  for  all  algorithms  at  the  ex¬ 
pense  of  a  decreased  sensitivity,  a  decreased  specificity,  and  an 
increased  number  of  decision  switches. 

For  practical  applications,  we  suggest  the  use  of  a  20-min 
AR-model  prediction  window  and  a  temperature  alert  threshold 
set  to  39  °C.  This  should  provide  a  sufficiently  high  threshold 
to  reduce  spurious  alerts,  while  allowing  for  useful  lead  time 
for  proactive  cooling  interventions  that  could  alter  the  course 
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TABLE  III 

Number  of  Events  for  Each  Subject  at  Different  Temperature  Thresholds 


Subjects 

Temperature  threshold  (°C) 

38.0 

38.1 

38.2 

38.3 

38.4 

38.5 

38.6 

38.7 

38.8 

38.9 

39.0 

39.1 

39.2 

39.3 

39.4 

39.5 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

2 

3 

3 

3 

3 

3 

3 

3 

3 

2 

1 

3 

3 

3 

3 

4 

3 

3 

2 

1 

1 

1 

4 

1 

1 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

5 

1 

1 

1 

1 

1 

6 

2 

2 

2 

2 

2 

1 

7 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

2 

2 

2 

1 

1 

8 

9 

1 

10 

2 

1 

1 

1 

1 

1 

1 

11 

3 

4 

4 

3 

3 

3 

3 

2 

1 

1 

1 

1 

1 

12 

1 

1 

13 

14 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

15 

3 

3 

2 

1 

1 

16 

17 

18 

1 

1 

1 

19 

3 

3 

2 

1 

1 

1 

1 

1 

1 

1 

20 

2 

2 

2 

1 

1 

1 

21 

4 

4 

3 

2 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

22 

1 

1 

1 

1 

Total 

36 

35 

32 

27 

24 

21 

18 

15 

13 

12 

8 

7 

7 

6 

5 

4 

of  the  clinical  outcome  [4],  We  recommend  two  possible  alert 
algorithms:  When  the  intent  is  to  prevent  the  occurrence  (or 
reoccurrence)  of  heat  injury  at  any  cost,  we  recommend  the  use 
of  Model+PI,  which,  in  addition  to  its  simplicity,  provides  ex¬ 
tremely  high  sensitivity  (~98%)  and  a  long  effective  prediction 
horizon  (29  min).  When  spurious  alerts  are  the  main  concern,  we 
recommend  the  use  of  SPRT,  which  provides  the  smallest  num¬ 
ber  of  decision  switches  with  reasonable  overall  performance. 

A  practical  limitation  of  the  proposed  concept  is  the  use  of  an 
invasive  sensor  to  measure  core  temperature.  We  are  currently 
developing  an  alternative  approach  that  combines  environmen¬ 
tal  and  activity  measurements  with  phenomenological  and  first- 
principle  models  to  estimate  core  temperature  and  obviate  the 
need  for  ingestion  of  a  temperature  pill.  We  are  also  in  the  pro¬ 
cess  of  integrating  the  Model-fPI  and  SPRT  algorithms  into  the 
Equivital  EQ-02  sensor  electronics  module  [24]  (Hidalgo,  Ltd.) 
and  assessing  the  performance  of  the  integrated  system  during 
military  training  exercises.  Together  with  the  work  described 
here,  these  efforts  shall  lead  to  a  hardware/software  system  for 
real-time  alerting  of  an  increasing  core  temperature  and  a  re¬ 
duced  incidence  of  heat  injuries. 

Appendix  A 

Optimizing  the  SPRT  Algorithm 

The  seven  parameters  of  the  SPRT  algorithm  (see 
Section  II-C;  the  two  weights  6  and  </>  in  (3)  and  the  five  pa¬ 
rameters  fiQ,  fii,  ax ,  A,  and  B)  were  estimated  for  a  subject  by 
minimizing  the  following  cost  function: 


where  td  and  tm  denote  the  time  of  event  onset  and  the  model- 
predicted  event  onset,  respectively,  both  in  minutes.  We  set  a 
30-min  window  before  the  event  onset  to  force  the  optimiza¬ 
tion  algorithm  to  provide  the  best-possible  effective  prediction 
horizon.  The  four  measures,  namely,  sensitivity,  specificity,  ef¬ 
fective  prediction  horizon,  and  number  of  decision  switches, 
are  described  in  Section  II-D.  The  number  of  decision  switches 
was  normalized  by  twice  the  number  of  true  events  because 
each  model-predicted  event  that  is  not  a  true  event  causes  two 
decision  switches.  In  other  words,  the  fourth  term  in  (Al) 
penalizes  model-predicted  events  that  were  not  true  events. 
We  used  the  Nelder-Mead  simplex  method  implemented  in 
MATLAB  version  7.14  {function  fminsearch)  to  estimate  the 
SPRT  parameters  that  minimize  (Al).  Our  initial  efforts  at  op¬ 
timizing  (Al)  without  weighting  the  cost  for  sensitivity  led  to 
poor  effective  prediction  horizons.  Hence,  we  performed  several 
optimizations  by  weighting  the  sensitivity  by  5,  10,  20,  and  30 
and  found  that  for  weights  beyond  10  the  algorithm  performance 
did  not  improve  substantially. 

Appendix  B 

Cross-Validation  Study  Table 

Table  III  shows  the  number  of  subjects  whose  core  tempera¬ 
ture  crossed  a  temperature  threshold  (from  38.0  to  39.5  °C,  with 
0. 1  °C  increments)  and  triggered  an  event.  It  also  shows  the  total 
number  of  distinct  events  for  each  temperature  threshold. 

Disclaimer 


J=  10(1  —  Sensitivity)  -f  (1  —  Specificity) 

I  o-30-tm  #of  decision  switches 
2x(#  of  events) 


The  opinions  and  assertions  contained  herein  are  the 
(Al)  private  views  of  the  authors  and  are  not  to  be  construed  as 
official  or  as  reflecting  the  views  of  the  U.S.  Army  or  of  the 
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U.S.  Department  of  Defense.  This  paper  has  been  approved  for 
public  release  with  unlimited  distribution. 
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